Vehicle localization refers to constructing and updating a map of an unknown environment while simultaneously keeping track of a vehicle's location within the environment. To determine a precise position of the vehicle, several inputs can be fused together, for example, odometry information, vehicle sensor information, and stored lane level maps. Based on these inputs, a vehicle pose can be determined. However, the integrity of the input data as well as the environment around the vehicle (e.g., lane curves, lane changes) present challenges to determining a reliable and precise vehicle pose and localization of the vehicle within lanes.